Most large scale institutions now rely on network based electronic mail systems to facilitate communication amongst their staff. In the last few years, "e-mail" has become as ubiquitous as the telephone as a vehicle of communication. It is not unusual for users of e-mail to generate dozens of messages per day. These messages will invariably deal with mundane matters such as meeting schedules, lunch orders and the like. In addition, e-mail will include mission critical communications and notifications going to the very heart of the business endeavors of the institution.
In a large sense, the volume of e-mail is now displacing communication that was otherwise recorded in hard copy--on paper--and, therefore, a permanent record. Paper recorded communications, while more expensive and time consuming to create and circulate, had the advantage of simple retention for later confirmation.
In fact, in several industries subject to governmental regulations regarding archiving records, significant rules and procedures have been instituted and mandated so that the paper records of communications relating to certain specific subjects are retained for select proscribed periods. An example of this exists in the securities industry. Companies involved with security transactions on behalf of customers, such as stock and bond trading, are required to maintain records of their communications with their customers and relating to their customer accounts.
In the past, maintaining these records and staying in compliance with the regulations involved a series of protocols relating to the retention of paper copy communications corresponding to time periods set by the regulators. For the big banks and brokerage houses, large stores of paper records were routinely generated. As their volume grew, the records were transferred to other forms of recorded media, such as microfiche and the like. This met the regulations, and substantially reduced costs. However, accessing stored records was a time consuming and difficult process, mostly because of the difficulty and expense in tracking and retrieving individual documents from a large volume of similarly formatted information.
With the advent of e-mail, many communications that are subject to the current retention regulations are now communicated electronically without a corresponding physical, paper copy. While some of the messages are printed to, hard copy and then stored, a vast majority are not, and thus exist in electronic storage media, often in a random ad hoc fashion. In particular, storage on a hard disk leaves the original message in a somewhat tenuous condition. Hard drives are prone to periodic failure and data loss. In addition, text based messages are subject to editing and thus there may be some question on whether the original message exists unaltered.
Perhaps more difficult are the problems associated with subsequent retrieval of e-mail based communications. There are several dimensions to this problem. The first relates to the volume of e-mail that is generated. It is enormous and growing rapidly. While text based e-mail is subject to common formatting protocols, recent e-mail volume includes a growing percentage of file attachments such as image files, spreadsheets, graphics, and the like. In the future, and as communication bandwidth increases, these attachments will further include multimedia elements such as audio clips and video segments. Accordingly, efforts to comply with current regulations cannot be simply handled by hard copy creation and storage.
In addition, storage and archiving requirements are often source, recipient, and/or content specific. That is, the length of time a message must be retained often depends on the parties to the communication and the subject matter of the message author. For example, client communications are often stored for periods up to ten years, while internal communications are stored for periods up to four years. Some messages will be accessed soon after generation, while others may only be accessed years after generation - in the course of some dispute resolution. Finally, the messages must be stored such that there is no question about subsequent alteration.
Presently e-mail is often handled by simply storing a log of all e-mail on a central server. A supervisor reviews random pieces of e-mail in the log on some periodic basis, by browsing the log to insure compliance with standing regulations. On a periodic basis (monthly) the log is purged of data that is more than, e.g. 30 days old. This information is then copied to optical disk (non-volatile storage).
This process has many shortcomings. Retrieval involves a huge manually intensive search effort. The process does not result in the non-alterable format demanded by most regulators. The process is time consuming and of questionable accuracy. No trail of the reviewing process is provided, thus making it difficult to establish compliance to regulators.
It was with an understanding of the problems outlined above that led to the present invention.